Hidden Bawls, Whispers, and Yelps: Can Text Convey the Sound of Speech, Beyond Words?

Whether a word was bawled, whispered, or yelped, captions will typically represent it in the same way. If they are your only way to access what is being said, subjective nuances expressed voice be lost. Since so much of communication carried by these nuances, we posit that if used as an accurate representation speech, embedding visual representations paralinguistic qualities into could help readers use them better understand speech beyond its mere textual content. This paper presents model for processing vocal prosody (its loudness, pitch, and duration) mapping dimensions typography (respectively, font-weight, baseline shift, letter-spacing), creating lost subtleties can embedded directly typographical form text. An evaluation out where participants were exposed this speech-modulated asked match originating audio, presented between similar alternatives. Participants (n=117) able correctly identify original audios with average accuracy 65%, no significant difference when showing modulations animated static Additionally, participants' comments showed their mental models varied widely.